3D Generic Elastic Models for 2D Pose Synthesis and Face Recognition

Pose, illumination, expression and the generalization of such effects to unseen face data samples are the fundamental problems faced in face recognition. The significant contribution of this thesis is the ability to match any two face images with a large pose angle variation. This approach utilizes a proposed 3D prior face model in order to cover a wide range of poses. To achieve this, a rapid 3D modeling scheme is proposed, called 3D Generic Elastic Model (GEM), which allows the synthesis of novel 2D images faster and more realistically than traditional 3D Morphable Model (3DMM) approaches used to date. In contrast, our work only requires the observed facial landmarks in a face image (see Appendix A for proposed work in robust facial landmarking and alignment using combined Active Shape and Active Appearance Models), coupled with the proposed 3D GEM depth-map generated from the USF Human-ID database. Although we only use a single GEM, we show that we can model a diverse set of 3D dense face shapes which provide visually accurate novel 2D pose synthesis of faces. Indeed, we show that our 3D models can be successfully applied not only to 2D pose synthesis but also to novel illumination synthesis. The proposed modeling approach is fully automatic, robust, and able to synthesize visually accurate 2D images in less than a few seconds, resulting in significant overall improvement over traditional 3DMMs that have been reported to take 4-5 minutes and require manual feature annotation by the user. Experimental results show that 2D faces, modeled by our proposed 3D GEMs, can effectively handle and match against large 3D pose changes. We use the CMU Multi-PIE database for evaluating the proposed work in this thesis and show that a simple cosine distance measure for face recognition is sufficient, even when trying to match face images with large pose variations. Our analysis concludes that 3D depth information of faces does not dramatically change across people (at least for the same ethnic group), indicating that depth information is not significantly discriminative for modeling 2D pose variability. Therefore, a 3D face model can be efficiently produced by using generic depth models that can be elastically deformed